Using Flux CD for Easier GitOps

May 29, 2019 · 52 min read

Front End Developer @ Tech4Biz Solutions Pvt Ltd.

Welcome to this guide! Here, you'll discover how to:

Set up a Flux CD on your DOKS cluster using the flux CLI.
Keep your Kubernetes cluster apps in sync with a Git repository (using GitOps principles).
Install and handle applications using Flux CD HelmReleases.
Set up and manage the Sealed Secrets controller with Flux CD to secure important information.

Let's make your journey into these concepts easy and enjoyable!

Once you've followed all the steps in this guide, your DOKS cluster will be equipped with Flux CD. Here's what it will do:

Keep things in sync: Flux CD will make sure your cluster stays updated by looking at the instructions in your GitRepository.
Manage applications smoothly: It will handle Helm releases effortlessly, paying attention to the details outlined in your HelmRelease.

In simpler terms, your DOKS cluster will be smartly organized and will always know what's happening thanks to Flux CD!

Flux CD is like a helpful guide for your computer setup. It makes sure everything in your system is up-to-date by checking a special place called Git. Think of Git as a master plan for your computer.With Flux CD, you can easily update and deliver your apps without any hassle. It's like having a smart assistant that knows how to take care of everything. It even understands the language of Kubernetes, which is like a super tool for managing computer programs.So, with Flux CD, your computer stays organized and you can control how your apps get installed and updated in a smooth and easy way!

Flux CD helps keep your computer organized by making sure everything is in the right place. This process is called reconciliation. It's like having a team of helpers that regularly checks your computer to match it with a special plan you've set up somewhere, like in a special folder (Git), a magic box (Helm or S3). In simpler terms, reconciliation is the way these helpers ensure your computer is always the way you want it to be – neat and tidy!

Flux CD gives you different tools to make sure your computer stays organized. Here are three ways it does that:

HelmRelease: Helps in checking and updating your computer's state using something called Helm releases.
Bucket: Assists in checking and updating your computer by looking at things stored in a special place called a bucket.
Kustomization: Uses a set of instructions to organize your computer, stored in a special folder (Git) or a specific storage location (S3 bucket).

In simpler terms, Flux CD offers various tools to make sure your computer stays organized and follows the plans you've set for it!

Flux CD uses special helpers called controllers. Each controller has a specific job:

Source Controller: Manages where you keep your plans, like Git, Helm, or a storage place. It fetches instructions to make sure your computer does what you want.
Helm Controller: Handles special updates for your computer known as Helm releases.
Kustomize Controller: Follows a set of rules for organizing your computer, ensuring it follows instructions stored in a special folder or storage place.

In simpler terms, these controllers work behind the scenes to keep everything organized and make sure your computer follows your plans!

Flux CD has a special tool called Source CRD (Custom Resource Definition). Think of it like a helper that grabs important information for your computer.This tool acts like a messenger – it goes to special places, like Git, Helm or a storage area, and brings back the instructions for how your computer should be set up. So, the Source CRD helps Flux get all the necessary details to keep your computer in the right state!

The Starter Kit uses a special tool called a Git repository to keep track of how your applications should be. It also uses something called HelmReleases to help with putting your applications on your computer.

Now, when it comes to keeping an eye on how your computer is doing, Flux CD has a friend called the Notification Controller. This friend can send you messages on platforms like Slack, Discord, or Microsoft Teams to let you know what's happening. It's like having a buddy who keeps you informed!

For watching over the overall health of your computer, Flux CD supports tools like Prometheus (official guide) and Loki. They're like doctors who check your computer to make sure everything is working smoothly!

Requirements:

Before we start, make sure you have the following:

A ready-to-go DOKS cluster: Think of this as your team of computer helpers. Follow the Starter Kit DOKS Setup Guide to get it set up.
A special place on the internet: This is like a digital notebook (GitHub repository and branch). It's where we'll keep important information for Flux CD and your computer programs.
A copying tool for the internet: We'll need a Git client to grab some helpful information. It's like a digital copy-paste tool.
A talking tool for your DOKS cluster: This is called Kubectl CLI. It helps you talk to your DOKS cluster, telling it what to do. Follow some simple instructions to connect it.
A special helper for Flux CD: Flux CLI is like a magic wand. It helps us deploy and talk to Flux CD, our smart assistant for computer programs.
A secret keeper: Kubeseal is a tool for keeping secrets safe. It helps us lock away important information, and it's friends with the Sealed Secrets Controller.

With these tools, you'll be all set to make your computer do amazing things!

Exploring Flux CD: Making Helm Releases Super Easy

Flux CD efficiently manages Helm releases, acting as a guide for setting up applications on your computer. It utilizes HelmRelease to understand and handle these instructions. Every HelmRelease relies on HelmRepository, a companion in this process. HelmRepository serves as a resource where Flux CD accesses specific guides, known as Helm charts, to install your applications. Think of HelmRepository as the place where Flux CD finds the instructions to make your apps work on your computer!

Now that you've tried installing things with Helm using commands (like in the Starter Kit), Flux CD has a similar way of doing it through its Helm Controller. Instead of typing commands, you create special instructions (manifests) using some codes like CRDs (HelmRepository, HelmRelease). These codes tell the Helm Controller to do the same tasks as the commands you used before (like 'helm install' or 'helm upgrade'). It's like giving Flux CD a set of rules to follow for installing and updating things on your computer!

Setting Up Special Places for Flux CD with HelmRepository

Flux CD uses something called HelmRepository to deal with Helm repositories. It's like a helper that goes to a special place on the internet to bring back the charts needed for your computer. You can think of HelmRepository as a set of instructions you give to Flux CD. These instructions are like telling Flux CD, "Hey, go to this specific place online and grab the charts we need." It's a simple way of making sure Flux CD knows where to find the important stuff for your computer!

Here's what a HelmRepository setup usually looks like:

apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: HelmRepository
metadata:
name: ambassador
namespace: flux-system
spec:
interval: 10m0s
url: https://app.getambassador.io

Let's break down what each part means:

<kind>: This tells Flux CD what type of setup we're using (like HelmRepository in the example).
<metadata.name>: It's the name you give to this HelmRepository setup (like 'ambassador' in the example).
<metadata.namespace>: Think of it as a special area for this HelmRepository (like 'flux-system' in the example).
<spec.interval>: This is like setting how often Flux CD should check for new things from the place we're getting our charts (similar to saying 'helm repo update' in the command line).
<spec.url>: It's the web address where Flux CD can find the charts we want to use for our computer.

So, in simple terms, it's like giving a Flux CD a note saying what type of setup, the name, the special area, how often to check, and where to find the charts.

Installing Apps Made Easy with HelmRelease

Flux CD uses a tool called HelmRelease to manage and install apps on your computer in the DOKS cluster. Think of it as a helpful guide that knows how to put apps on your computer in a simple way. This tool not only installs apps but also handles updates and fixes if something goes wrong. It's like HelmRelease is a superhero for your computer, making sure everything stays just right with your apps!

Every HelmRelease needs to know where to find the app it's installing. It uses different sources to do this:

HelmRepository: This is like a store for Helm charts. HelmRelease can go here to get the chart it needs.
GitRepository: Think of this as a library where HelmRelease can find charts stored in Git repositories.
S3 Bucket: This is a special storage place where HelmRelease can find charts if they're kept in S3 compatible buckets.

So, HelmRelease uses these sources to make sure it finds the right chart for your app!

What a HelmRelease Configuration Looks Like:

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: ambassador-stack
namespace: flux-system
spec:
chart:
  spec:
    chart: edge-stack
    sourceRef:
      kind: HelmRepository
      name: ambassador
    version: 7.3.2
install:
  createNamespace: true
interval: 1m0s
targetNamespace: ambassador

Breaking Down the HelmRelease Configuration:

<kind>: This tells Flux CD it's a type of setup called HelmRelease.

<metadata.name>: It's like giving a name to this specific HelmRelease setup, such as 'ambassador-stack'.

<metadata.namespace>: Think of this as a special area where your HelmRelease works, like 'flux-system'.

<chart.spec.chart>: Specifies the name of the app's instruction manual (chart) to get from the HelmRepository, like 'ambassador'.

<chart.spec.sourceRef.kind>: Tells Flux CD to use a resource called HelmRepository to find the app's instruction manual (it's like a helper for HelmRelease).

<chart.spec.sourceRef.name>: This is the name of that HelmRepository helper.

<chart.spec.version>: Specifies which version of the app's instruction manual to use.

<spec.install.createNamespace>: Instructs Flux CD if it needs to create a special area for the app before installing it (like 'helm install --create-namespace' in commands).

<spec.interval>: Tells Flux CD how often it should check and make sure everything is set up correctly.

<spec.targetNamespace>: Specifies where the app's instruction manual should be placed, similar to saying 'helm install --namespace <name>' in commands.

Customizing Apps with HelmRelease:

With Flux CD's HelmRelease, you can change how apps behave by tweaking their settings. It's like adjusting the settings on your favorite game or app!

You can do this in two ways:

Using a values file: This is like giving Flux CD a file with all the changes you want to make. It's similar to saying 'helm install <name> <chart> -f values.yaml' in commands.
Setting each value individually: Instead of using a file, you can tell Flux CD each change you want to make one by one. It's like saying 'helm install <name> <chart> --set <key> <value>' in commands.

So, with HelmRelease, you have the power to customize your apps just the way you like!

Flux CD gives you two ways to change how your apps work:

Inline Overrides (using <spec.values>): It's like telling Flux CD exactly what changes you want, similar to giving it a note with all the adjustments you need. This is the equivalent of saying 'helm install -f <values_file>' in commands.
Individual Overrides (using <spec.valuesFrom>): Instead of giving Flux CD a note, you can talk to it one-on-one and say each change you want. It's like saying 'helm install --set <key> <value>' in commands, but in a more detailed way.

So, with Flux CD, you have these cool options to make your apps work just the way you want!

How to Use <spec.values> in Flux CD:

...
spec:
values:
  loki:
    enabled: true
    config:
      schema_config:
        configs:
          - from: "2020-10-24"
            store: boltdb-shipper
            object_store: aws
            schema: v11
            index:
              prefix: index_
              period: 24h
...

Understanding <spec.values> Configuration:

Think of <spec.values> as a special note where you write instructions for your app, such as what color it should use or how fast it should run.

But here's a caution: if you put sensitive information like passwords or API tokens directly in this note, it's not safe. Flux CD is smart, though. It provides a solution! Instead of putting secrets in the note, you can keep them in a secret vault called Kubernetes Secrets or ConfigMaps. These are like hidden safes where important things stay protected. Flux CD then knows exactly where to find these secrets when needed, keeping everything secure!

How to Use <spec.valuesFrom> with Kubernetes Secret: This is a way of telling Flux CD to use a secret vault called Kubernetes Secret, and it expects you to set up a secret named 'do-credentials' before using it.

spec:
valuesFrom:
  - kind: Secret
    name: do-credentials
    valuesKey: spaces_access_key
    targetPath: aws.access_key_id

Understanding the Configuration Details:"

<spec.valuesFrom.kind>: This tells Flux CD the type of secret vault or storage place to look into, like a hidden safe or a special storage box (in this case, it's a Kubernetes Secret, but it could also be a ConfigMap).
<spec.valuesFrom.name>: It's like providing the name of the secret vault or storage box Flux CD should check for important information. This vault or box should be in the same special area as the HelmRelease.
<spec.valuesFrom.valuesKey>: Think of this as a specific note inside the vault or box where Flux CD can find something important. If not set, Flux CD assumes it's a general set of instructions named 'values.yaml'.
<spec.valuesFrom.targetPath>: This is like instructing Flux CD where to put the important information it finds. If not set, Flux CD puts the information in the main place.

In simpler terms, it's like telling Flux CD where to look (kind and name), what specific information to find (valuesKey), and where to put it (targetPath).

When using Flux CD to manage your apps in Kubernetes (think of it like a helpful guide for your computer), there are two types of information your apps might need:

Basic Info: Like the everyday details your app uses, such as settings. You can keep these in a place called spec.values.
Secret Info: Some things are secret, like passwords. You don't want these out in the open. Instead, you can point to a secret stash using spec.valuesFrom.

When Flux CD sets up your app, it brings together both the basic and secret info. It's like making sure your app has everything it needs.

But, because you shouldn't just put secret things out where anyone can see (like leaving your diary on a park bench), in a GitOps way of working (managing your code with Git), you use something called Sealed Secrets. It's like putting your secrets in a super-secure envelope so that only your app can open it when needed.

Keeping Secrets Safe with Sealed Secrets: A Beginner's Guide

Here's how it works:

Locking Secrets: Sealed Secrets locks up your secrets, making them like a secret code only your computer can understand.
Safe Storage: You can put these locked-up secrets in a special storage place called Git. It's safe, even if it's a shared space, because nobody can open the locked-up secrets without the right key.
Flux CD Help: When you need these secrets for your apps, Flux CD, your computer helper, sees the locked-up secrets in Git. It creates a special key to open the lock, revealing your original secrets.
Unlocking for Apps: Another helper, the Sealed Secrets Controller, notices these unlocked secrets and turns them back into regular codes that your apps can use.

So, Sealed Secrets helps keep your secrets safe, even in shared spaces, and ensures your apps can still use them without any problems. It's like having a secret code and a key to unlock it whenever you need it.

Next, let's see how to put a Flux CD on your DOKS cluster.

Getting Started with Flux CD

Setting up Flux CD is like having a helpful assistant for your computer that makes managing your apps easier. You use a tool provided by Flux CD to run a command called "flux bootstrap." This command tells your computer to set up a Flux CD on your Kubernetes cluster, enhancing its ability to handle your apps smoothly. Flux CD keeps track of your app details in a special place called Git, similar to a digital to-do list.

So, when you run the command, you're not only setting up a Flux CD but also saying, "Look at this list in Git to understand my apps." The good part is, if you need to run the command again, it's safe. It won't cause any issues, making it easy to remind your computer to get ready as many times as you need.

When setting up Flux CD on your existing DOKS cluster, it's like getting your computer ready for a helpful assistant. Here's how you do it:

Create a Special Key: Make a special key for your computer called a Personal Access Token (PAT). This key allows your computer to do specific tasks, like making a list of your apps.
Share the Key with Your Computer: Once you have this special key, you share it with your computer. It's like saying, "Hey, computer, here's a key that lets you make lists. Use it!" Just replace some placeholders with the actual key.

In simpler terms, you're giving your computer permission to make a list of your apps, and you're just making sure it has the right key to do it.

export GITHUB_TOKEN=<YOUR_PERSONAL_ACCESS_TOKEN_HERE>

3. Set up a folder in your own GitHub account by running a special command (don't forget to fill in your own info instead of the placeholders).

flux bootstrap github --owner=<YOUR_GITHUB_USER> --repository=<YOUR_GITHUB_REPOSITORY_NAME> --path=clusters/dev --personal

Let's break down what these instructions mean:

--owner: This is like telling your computer who you are on GitHub, just saying, "It's me!
--repository: Think of this as asking your computer to make a special folder for Flux CD on GitHub. If the folder isn't there, your computer will create it.
--path: Imagine this as picking a specific spot inside that folder to keep all the important stuff. Flux CD will keep an eye on any changes happening in this particular spot.

When you use the "flux bootstrap" command, it sets up everything Flux CD needs to work smoothly. It'll make a special place for Flux CD on GitHub if it's not already there, and it'll start getting Flux CD ready to go. After it's done, you'll find a bunch of files in your GitHub folder. These files, written in a language called YAML, make sure Flux CD has everything it needs to do its job with your apps. So, think of it as saying, "Hey computer, get everything ready for Flux CD to work well!

Next, you can do some quick checks by:

flux check

The result will appear something like this:

Output► checking prerequisites
✔ Kubernetes 1.21.9 >=1.20.6-0
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v0.17.2
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v0.21.1
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v0.22.3
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v0.21.2
✔ all checks passed

After that, take a look at all Flux CD resources by:

flux get all

You'll see something like this (just a note: you might notice parts like "gitrepository/flux-system" and "kustomization/flux-system" doing their thing, which means they're fetching the latest updates from your main branch).

OutputNAME                            READY   MESSAGE                         REVISION        SUSPENDED 
gitrepository/flux-system             True    Fetched revision: main/6e9b41b  main/6e9b41b    False 

NAME                            READY   MESSAGE                         REVISION        SUSPENDED 
kustomization/flux-system       True    Applied revision: main/6e9b41b  main/6e9b41b    False

If you ever need to figure out what Flux CD is up to or fix something, you can check out its logs by:

flux logs

You'll see something like this:

Output...
2022-03-17T10:47:21.976Z info Kustomization/flux-system.flux-system - server-side apply completed 
2022-03-17T10:47:22.662Z info Kustomization/flux-system.flux-system - server-side apply completed 
2022-03-17T10:47:22.702Z info Kustomization/flux-system.flux-system - Reconciliation finished in 9.631064885s, next run in 10m0s 
2022-03-17T10:47:19.167Z info GitRepository/flux-system.flux-system - Discarding event, no alerts found for the involved object 
2022-03-17T10:47:22.691Z info Kustomization/flux-system.flux-system - Discarding event, no alerts found for the involved object 
2022-03-17T10:47:22.709Z info Kustomization/flux-system.flux-system - Discarding event, no alerts found for the involved object 
2022-03-17T10:47:19.168Z info GitRepository/flux-system.flux-system - Reconciliation finished in 7.79283477s, next run in 1m0s 
2022-03-17T10:48:20.594Z info GitRepository/flux-system.flux-system - Reconciliation finished in 1.424279853s, next run in 1m0s 
...

Lastly, make sure Flux CD knows about your Git repository by:

kubectl get gitrepositories.source.toolkit.fluxcd.io -n flux-system

The result will be something like this (check the URL column value - it should lead to your Git repository and make sure the READY state is set to True):

OutputNAME         URL                                                       READY  STATUS                             AGE
flux-system  ssh://git@github.com/test-starterkit/starterkit_fra1.git  True   Fetched revision: main/6e9b41b...  9m59s

You'll also find several Flux CD system files in your Git repository:

In the next step, we'll set up your Git repository in a way that Flux CD can understand. Flux CD pays attention to changes in a specific folder that you told it about when you ran the "flux bootstrap" command, and for our tutorial, it's the "clusters/dev" folder. Inside this folder, you have the freedom to create other folders or files to keep everything neat and organized. Flux CD will carefully go through all the files and folders inside "clusters/dev" to find what it needs.

You could just put everything directly into the "clusters/dev" folder, but it's a good habit to keep things well-organized and follow a set of rules for naming. This helps you stay organized and reduces confusion down the road.

Setting up Your Git Repository for Flux CD - Step by Step

In this step, you'll find out how to arrange your Git repository, which Flux CD uses to keep your DOKS cluster in sync. To keep things simple, we're using what's called a "mono repo" structure, where all your important stuff is in one place - think of it like a big box. Specifically, we use a folder named "clusters/dev" to store everything related to your apps. If you want to explore other ways to organize your Git repository, you can check out the official Flux CD documentation for some tips. But for now, we're keeping it easy with this one-folder approach.

Make sure you do these steps in order:

Get Your Flux CD Git Repository: Start by copying your Flux CD Git repository. It's like making a duplicate of your big box where all the important things are stored for looking after your DOKS cluster. Just fill in your details instead of the <> symbols.
```
git clone https://github.com/<YOUR_GITHUB_USER>/&lt;YOUR_GITHUB_REPOSITORY_NAME>.git
```
Here's what those placeholders mean:
1. <YOUR_GITHUB_USER>: This is where you put your username on GitHub, the same one you used when you set up Flux CD.
2. <YOUR_GITHUB_REPOSITORY_NAME>: This is the name of the special folder on GitHub where Flux CD keeps everything for your DOKS cluster. Remember, you set this up when you ran the "flux bootstrap" command.

So, when you see these placeholders, just replace them with your own GitHub username and the name of the repository you created for your DOKS cluster. It's like filling in the blanks with your own information.

2. Next, go to the folder where you copied your Flux CD Git repository, and switch to the right branch, usually called "main".

3. Now, it's time to set up the folders to keep all the important files for Flux CD, like HelmRepository, HelmRelease, and SealedSecret manifests for each part of the Starter Kit. To do this, replace the FLUXCD_SYNC_PATH variable with the location where Flux CD checks for changes. In our Starter Kit, we use the "clusters/dev" folder path for this.

FLUXCD_SYNC_PATH="clusters/dev"
FLUXCD_HELM_MANIFESTS_PATH="$ {FLUXCD_SYNC_PATH}/helm"
mkdir -p "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories" 
mkdir -p "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases" 
mkdir -p "$ {FLUXCD_HELM_MANIFESTS_PATH}/secrets"

Finally, let's make sure we don't include any secret information in your repository. Open the folder where you copied your Flux CD Git repository, and add a file named ".gitignore." You can use any text editor you like. Then, copy and paste the following text into that file. The example below follows the naming style of our Starter Kit:

Output# Ignore all YAML files containing the '-values-' string.
*-values-*.yaml

# Do not ignore sealed YAML files.
!*-sealed.yaml

Once you complete all the steps in this tutorial, your Git repository should look something like this:

├── README.md
├── clusters
│   └── dev
│       ├── flux-system
│       │   ├── gotk-components.yaml
│       │   ├── gotk-sync.yaml
│       │   └── kustomization.yaml
│       └── helm
│           ├── releases
│           │   ├── ambassador-stack-v7.3.2.yaml
│           │   ├── loki-stack-v2.6.4.yaml
│           │   ├── prometheus-stack-v35.5.1.yaml
│           │   ├── sealed-secrets-v2.4.0.yaml
│           │   └── velero-v2.29.7.yaml
│           ├── repositories
│           │   ├── ambassador.yaml
│           │   ├── grafana.yaml
│           │   ├── prometheus-community.yaml
│           │   ├── sealed-secrets.yaml
│           │   └── vmware-tanzu.yaml
│           └── secrets
│               ├── do-api-credentials-sealed.yaml
│               ├── do-spaces-credentials-sealed.yaml
│               └── prometheus-stack-credentials-sealed.yaml
└── pub-sealed-secrets-dev-cluster.pem

Next, we'll set up the necessary files for Flux CD for each part of the Starter Kit. Then, we'll carefully check each file and save them to your Git repository. This repository is where Flux CD looks to keep your DOKS cluster up to date. If there's any private information, like passwords, we'll keep them safe by putting them in a special encrypted file called a Kubernetes Secret. This file will also be stored in your Git repository.

First, we'll use a tool called the Flux CLI to create some files. This will help you get used to working with them. After that, we'll use some files that are already made and provided by the Starter Kit. This will make things faster and easier.

We're going to start with something called the Sealed Secrets Helm release. It's important because the other parts of the Starter Kit need it to work.

Starting the Sealed Secrets Setup

In this step, you'll use a tool called Flux CLI to create some files. These files are needed to set up something called Sealed Secrets Helm release. After that, Flux will handle installing the Sealed Secrets Controller on your DOKS cluster.

Follow these steps to make the necessary files for the Sealed Secrets Helm release:

Go to the folder where you cloned your Flux CD Git repository. Make sure the folder structure needed for this tutorial is set up and that you've set the FLUXCD_HELM_MANIFESTS_PATH environment variable.

Create a file called Sealed Secrets HelmRepository manifest for Flux

flux create source helm sealed-secrets --url="https://bitnami-labs.github.io/sealed-secrets" --interval="10m" --export > "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/sealed-secrets.yaml"

Understanding Command Options

The output looks like this (you might see it organized in a similar way to what we talked about in Using HelmRepository CRD to Specify Helm Repositories):

apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: HelmRepository
metadata:
name: sealed-secrets
namespace: flux-system
spec:
interval: 10m0s
url: https://bitnami-labs.github.io/sealed-secrets

Next, get the Starter Kit's values file for Sealed Secrets. Before you use it, take a look at the file to see if there are any placeholders, like <> symbols, that you need to fill in with the right information.

SEALED_SECRETS_CHART_VERSION="2.4.0"
curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/06-kubernetes-secrets/assets/manifests/sealed-secrets-values-v$ {SEALED_SECRETS_CHART_VERSION}.yaml" > "sealed-secrets-values-v$ {SEALED_SECRETS_CHART_VERSION}.yaml"

Now, make the Sealed Secrets HelmRelease file for Flux CD. By default, the Kubeseal CLI expects to find the controller in a specific place and named in a certain way. So, we'll change the release name using some special instructions. This isn't necessary, but it's often done because the default location might only be available to certain advanced users, like administrators.

SEALED_SECRETS_CHART_VERSION="2.4.0"
flux create helmrelease "sealed-secrets-controller" --release-name="sealed-secrets-controller" --source="HelmRepository/sealed-secrets" --chart="sealed-secrets" --chart-version "$ SEALED_SECRETS_CHART_VERSION" --values="sealed-secrets-values-v$ {SEALED_SECRETS_CHART_VERSION}.yaml" --target-namespace="flux-system" --crds=CreateReplace --export > "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/sealed-secrets-v$ {SEALED_SECRETS_CHART_VERSION}.yaml"

Explaining How to Create HelmRelease Manifest with Simple Commands

--release-name: This is the name you want to give to your Helm release. If you don't specify one, it will use a default name based on the target namespace and HelmRelease name.
--source: This tells Flux where to find the chart. It should be in the format kind/name.namespace. The kind can be HelmRepository, GitRepository, or Bucket.
--chart: This is the name of the Helm chart you want to use.
--chart-version: Specify the version of the Helm chart you want to use.
--values: This is the path to a file containing values for configuring the Helm chart.
--target-namespace: This is the namespace where you want to install this Helm release.
--crds: This option determines what to do with Custom Resource Definitions (CRDs). You can choose to skip, create, or create/replace them.
--export: This option makes the command output the information in YAML format on the screen

The result appears like this (you can see it follows a similar pattern as explained in Using HelmRelease CRD to Install Helm Charts):

---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: sealed-secrets-controller
namespace: flux-system
spec:
chart:
  spec:
    chart: sealed-secrets
    sourceRef:
      kind: HelmRepository
      name: sealed-secrets
    version: 2.4.0
interval: 1m0s
releaseName: sealed-secrets-controller
targetNamespace: flux-system
install:
  crds: Create
upgrade:
  crds: CreateReplace
values:
  ingress:
    enabled: false

Lastly, upload the Git changes to the remote branch:

SEALED_SECRETS_CHART_VERSION="2.4.0"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/sealed-secrets.yaml"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/sealed-secrets-v$ {SEALED_SECRETS_CHART_VERSION}.yaml"
git commit -am "Adding Sealed Secrets manifests for Flux CD"
git push origin

After finishing the steps above, Flux CD will begin updating your DOKS cluster (usually within a minute, or as per the default interval). If you don't want to wait, you can manually trigger the update by:

flux reconcile source git flux-system

After a short while, take a look at the Flux CD Sealed Secrets Helm release:

flux get helmrelease sealed-secrets-controller

The result resembles:

NAME                        READY   MESSAGE                                 REVISION        SUSPENDED 
sealed-secrets-controller   True    Release reconciliation succeeded        2.4.0            False

Check the "READY" column, it should show "True". The "MESSAGE" column tells you about the status of the update, and you'll also see a number in the "REVISION" column, which is the version of the Helm chart. Sometimes updates, especially for complex things like the Prometheus stack, can take a while, so don't worry if it's not instant—just be patient.

The MESSAGE column will show "Reconciliation in progress" while the HelmController is installing the specified Helm chart. If there's an issue, you'll see a different message explaining what went wrong, so it's important to keep an eye on the Helm release status.
You can use the --watch flag like this: flux get helmrelease <name> --wait, to wait until the installation finishes. Just know that when using this flag, your terminal will be busy until the default timeout of 5 minutes passes (you can change this with the --timeout flag).
If something doesn't go as planned, you can search through the Flux logs and focus only on HelmRelease messages:

flux logs --kind=HelmRelease

If you can't find enough information in the Flux logs, you can use the describe command on the HelmRelease using kubectl like this:

kubectl describe helmrelease sealed-secrets-controller -n flux-system

Exporting the Sealed Secrets Controller Public Key

To encrypt secrets, you'll need the public key created by the Sealed Secrets Controller when Flux CD deployed it in your DOKS cluster.

Here's what to do:

Go to the folder where you cloned your Flux CD Git repository.Follow these steps (replace any <> placeholders as needed):
```
kubeseal --controller-namespace=flux-system --fetch-cert > pub-sealed-secrets-<YOUR_DOKS_CLUSTER_NAME_HERE>.pem
```
If you're having trouble with the kubeseal certificate fetch command, or if you get an empty or invalid certificate file, you can try these steps to fix it:
Open a new terminal window. Use this command to make the Sealed Secrets Controller service accessible on your computer (you can press CTRL - C to stop it after you've fetched the public key):
```
kubectl port-forward service/sealed-secrets-controller 8080:8080 -n flux-system
```

After that, return to your original terminal and use the command to get the public key (make sure to replace any <> placeholders as needed):

curl --retry 5 --retry-connrefused localhost:8080/v1/cert.pem > pub-sealed-secrets-<YOUR_DOKS_CLUSTER_NAME_HERE>.pem

Lastly, upload the public key file to your remote Git repository. It's safe to do this because the public key alone can't be used to decrypt anything; the private key, which is needed for decryption, is securely stored in your DOKS cluster. Run the following commands and replace any placeholders like <> with the correct information:

git add pub-sealed-secrets-<YOUR_DOKS_CLUSTER_NAME_HERE>.pem
git commit -m "Adding Sealed Secrets public key for cluster <YOUR_DOKS_CLUSTER_NAME_HERE>"
git push origin

Note: In this tutorial, we're using a special place called the flux-system namespace to keep some secrets in Kubernetes. Just make sure that regular users or applications can't get into this place by setting up RBAC (Role-Based Access Control).

Setting Up Cert-Manager for Wildcard Certificates

If you want to use wildcard certificates in your cluster or need proper TLS termination for the Nginx Ingress Controller, you'll need to install Cert-Manager.

Here are the steps to follow:

Go to the folder where you saved your Flux CD Git repository. Make sure you have the right folders set up for this tutorial, and that you've set the FLUXCD_HELM_MANIFESTS_PATH environment variable (you can find instructions for this in Step 4 - Cloning the Flux CD Git Repository and Preparing the Layout).

Next, download the Jetstack HelmRepository manifest file from the Starter Kit Git repository.

curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/repositories/jetstack.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/jetstack.yaml"

Next, get the Cert-Manager HelmRelease manifest file from the Starter Kit Git repository:

CERT_MANAGER_CHART_VERSION="1.8.0"
curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/releases/cert-manager-v$ {CERT_MANAGER_CHART_VERSION}.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/cert-manager-v$ {CERT_MANAGER_CHART_VERSION}.yaml"

Next, open the downloaded HelmRelease manifest file in a text editor. It's a good idea to use one with support for YAML linting, like VS Code. Make any necessary adjustments to fit your requirements. If you see placeholders like <>, be sure to replace them with the correct information.
```
CERT_MANAGER_CHART_VERSION="1.8.0"
code "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/cert-manager-v$ {CERT_MANAGER_CHART_VERSION}.yaml"
```

Lastly, upload the Git changes to the remote repository:

CERT_MANAGER_CHART_VERSION="1.8.0"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/jetstack.yaml"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/cert-manager-v$ {CERT_MANAGER_CHART_VERSION}.yaml"
git commit -am "Adding Cert-Manager manifests for Flux CD"
git push origin

Once you finish the steps above, Flux CD will begin updating your DOKS cluster. It usually takes about a minute, depending on the default interval. If you're in a hurry, you can manually trigger the update by:

flux reconcile source git flux-system

After a short while, please check the status of the HelmRelease:

flux get helmrelease cert-manager

The output will look like this:

OutputNAME                READY   MESSAGE                                 REVISION       SUSPENDED 
cert-manager              True    Release reconciliation succeeded        v1.6.1         False

Look at the "READY" column, it should say "True". The "MESSAGE" column tells you how the update is going, and you'll see a number in the "REVISION" column, which is the version of the Helm chart. Some updates, like for the Prometheus stack, might take longer, so don't worry if it's not finished right away—just be patient.

The MESSAGE column shows 'Reconciliation in progress' while Flux CD is setting up the Helm chart. If there's a problem, you'll see a different message explaining it, so it's important to check the Helm release status.
You can use the --watch flag, like this: flux get helmrelease <name> --wait, to wait until everything's done. Just know that your terminal will be busy until it finishes (you can change the timeout if needed).
If something goes wrong, you can search the Flux logs for HelmRelease messages.

flux logs --kind=HelmRelease

If you can't find enough information in the Flux logs, you can use a command called "describe" on the helmrelease using kubectl like this:

kubectl describe helmrelease cert-manager -n flux-system

Next, you'll make Flux CD files for the Ambassador (or Nginx) ingress.

Setting up the Ingress Controller with Helm

In this step, you'll use ready-made files to set up your chosen Ingress Controller with Flux CD. Then, Flux will start installing the Ingress Controller on your DOKS cluster.

You have two choices for setting up your Ingress Controller, which is like a traffic director for your web applications:

Ambassador Ingress Helm Release
Nginx Ingress Helm Release

These are tools to help manage how internet traffic gets directed to your apps, kind of like traffic signs for your website.

Follow these easy steps to get started:

Navigate to Your Flux CD Git Repository: Go to the folder where you stored your Flux CD Git repository. Make sure the required folders are set up correctly (if you're unsure, check Step 4 of the tutorial for details).

Fetch the HelmRepository File: Now, grab the file that tells Flux CD how to handle Helm charts. Choose the option that matches the Ingress Controller you want (either Ambassador or Nginx). This file helps set up the necessary configurations.

That's it! You're on your way to configuring Flux CD for your project.

Ambassador Ingress:

curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/repositories/ambassador.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/ambassador.yaml"

NginxIngress:

curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/repositories/kubernetes-community-nginx.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/kubernetes-community-nginx.yaml"

3. Please get the file that sets up HelmRelease from the Starter Kit Git repository. Choose the one that matches the Ingress Controller you want to install and set up

Ambassador Ingress:

AMBASSADOR_CHART_VERSION="7.3.2"
curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/releases/ambassador-stack-v$ {AMBASSADOR_CHART_VERSION}.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/ambassador-stack-v$   {AMBASSADOR_CHART_VERSION}.yaml"

Nginx Ingress:

NGINX_CHART_VERSION="4.1.3"
curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/releases/nginx-v$ {NGINX_CHART_VERSION}.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/nginx-v$ {NGINX_CHART_VERSION}.yaml"

4. Now, open the HelmRelease file you downloaded using an editor like VS Code. Look for specific parts you may need to change, like anything in <>. Make sure to update these parts based on your preferences. If there are placeholders, replace them with the information you want to use. Just focus on making the file suit your needs.

Ambassador Ingress:

AMBASSADOR_CHART_VERSION="7.3.2"
code "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/ambassador-stack-v$ {AMBASSADOR_CHART_VERSION}.yaml"

Nginx Ingress:

NGINX_CHART_VERSION="4.1.3"
code "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/nginx-v$ {NGINX_CHART_VERSION}.yaml"

Note: Take a look at the HelmRelease file to see how it handles fixing things when there's a problem during installation or upgrade. If you want more details on how it works, you can visit the 'Configuring Failure Remediation' page for additional information about the options available in Flux CD for dealing with issues during Helm install or upgrade.

...
install:
createNamespace: true
remediation:
  retries: 3
upgrade:
remediation:
  retries: 3
...

5. At the end, save your changes in Git by committing them to the remote branch. Choose the option that matches the Ingress Controller you want to install and set up.

Ambassador Ingress:

AMBASSADOR_CHART_VERSION="7.3.2"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/ambassador.yaml"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/ambassador-stack-v$ {AMBASSADOR_CHART_VERSION}.yaml"
git commit -am "Adding Ambassador manifests for Flux CD"
git push origin

Nginx Ingress:

NGINX_CHART_VERSION="4.1.3"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/kubernetes-community-nginx.yaml"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/nginx-v$ {NGINX_CHART_VERSION}.yaml"
git commit -am "Adding Nginx manifests for Flux CD"
git push origin

After you've done the earlier steps, Flux CD will automatically check and update your DOKS cluster in about a minute (if using the default setting). If you're in a hurry and don't want to wait, you can manually trigger this process by following these steps:

flux reconcile source git flux-system

After waiting a bit, check the status of the HelmRelease. Choose the option that matches the Ingress Controller you installed.

Ambassador Ingress:

flux get helmrelease ambassador-stack

The result will appear something like this:

OutputNAME                    READY   MESSAGE                                 REVISION       SUSPENDED
ambassador-stack              True    Release reconciliation succeeded        7.3.2          False

Nginx Ingress:

flux get helmrelease ingress-nginx

The result will appear something like this:

OutputNAME                 READY   MESSAGE                                 REVISION       SUSPENDED
ingress-nginx              True    Release reconciliation succeeded        4.1.3          False

Check the 'READY' column, it should show 'True'. The 'MESSAGE' column will tell you about the status of the reconciliation, and you'll also see a 'REVISION' number which represents the version of the Helm chart. Keep in mind that some installations, like Prometheus stack, might take longer, so just be patient.

While the HelmController is busy installing the Helm chart, the 'MESSAGE' column will show 'Reconciliation in progress'. If anything goes wrong, you'll see a different message explaining the issue. So, keep an eye on the Helm release status.

If you want to wait for the installation to finish, you can use the command 'flux get helmrelease <name> --wait'. Just know that in this mode, your terminal will be busy until it finishes or reaches the default timeout of 5 minutes (you can change this with the --timeout flag).

If there's a problem, you can check the Flux logs for HelmRelease messages. Here's how to do it:

flux logs --kind=HelmRelease

If you can't find enough information in the Flux logs, you can use the 'describe' command with 'kubectl' on the HelmRelease. Here's how:

kubectl describe helmrelease ingress-nginx -n flux-system

To get more details about how the Ingress Controller is doing, check out the tutorial for either Ambassador Ingress or Nginx Ingress.

Now, let's move on to setting up Flux CD manifests for the Prometheus stack.

Setting up Prometheus Stack with Flux CD

In this step, we're going to use ready-made instructions to create the Prometheus Helm release with Flux CD. Flux will then take care of installing Prometheus on your DOKS cluster. The Prometheus stack also brings Grafana along, and we'll set up the administrator credentials so you can access the dashboards. We'll show you how to use the kubeseal CLI along with the Sealed Secrets Controller to protect sensitive information stored in Kubernetes Secrets. Finally, we'll explore how the Flux CD HelmRelease manifest helps us link to Grafana credentials safely stored in Kubernetes Secrets.

To prepare for installing Prometheus, follow these steps:

hange Directory: Go to the directory where you cloned your Flux CD Git repository. Make sure everything's set up correctly according to the tutorial's directory structure. If you're unsure, check Step 4 for guidance.

Fetch Prometheus HelmRepository Manifest: Get the Prometheus HelmRepository manifest from the Starter Kit Git repository. This will give you the necessary files to set up Prometheus.

curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/repositories/prometheus-community.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/prometheus-community.yaml"

3. First, make sure you have Sealed Secrets, a tool for keeping your secrets secure. Then, obtain a special key known as the Sealed Secrets public key as per the given instructions. Now, secure your Grafana dashboard password by using Kubernetes, a tool for managing things digitally. Create a secure space, place your password inside, and lock it with the special key. This ensures that only authorized entities can access your password, providing a layer of protection for your sensitive information.

SEALED_SECRETS_PUB_KEY="<YOUR_SEALED_SECRETS_PUB_KEY_NAME_HERE>"
  GRAFANA_ADMIN_PASSWORD="<YOUR_GRAFANA_ADMIN_PASSWORD_HERE>"
  kubectl create secret generic "prometheus-stack-credentials"       --namespace flux-system       --from-literal=grafana_admin_password="$ {GRAFANA_ADMIN_PASSWORD}"       --dry-run=client -o yaml | kubeseal --cert="$ {SEALED_SECRETS_PUB_KEY}"       --format=yaml > "$ {FLUXCD_HELM_MANIFESTS_PATH}/secrets/prometheus-stack-credentials-sealed.yaml"

The command has a few parts explained below:

--namespace: This is like telling the computer where to store the secret. It's a specific place called a namespace in Kubernetes.

--from-literal: Imagine putting a secret code directly into the computer. This part helps create a secret by using a simple value, like your Grafana admin password.

--dry-run=client: Think of this as practicing before doing the real thing. It's like checking if everything works without actually making any changes. This helps make sure everything is set up correctly.

4. Now, go and get the file that sets up Prometheus using a tool called Helm. You can find this file in the Starter Kit Git repository.

PROMETHEUS_CHART_VERSION="35.5.1"
curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/releases/prometheus-stack-v$ {PROMETHEUS_CHART_VERSION}.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/prometheus-stack-v$ {PROMETHEUS_CHART_VERSION}.yaml"

5. Now, let's take a look at the file we just downloaded for setting up Prometheus using Helm. You can use a program like VS Code (or any text editor you prefer). Make sure to replace any placeholders enclosed in <> brackets with the specific information you need.

PROMETHEUS_CHART_VERSION="35.5.1"

code "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/prometheus-stack-v$ {PROMETHEUS_CHART_VERSION}.yaml"

Note: Now, let's find out how the secret named 'prometheus-stack-credentials' is used in the Prometheus setup. Look at the part of the code below to see how it's mentioned:

Lastly, save your changes in Git and upload them to the main online storage space, which is like sharing your work with others. This helps keep everyone on the same page.

LOKI_CHART_VERSION="2.6.4"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/grafana.yaml"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/loki-stack-v$ {LOKI_CHART_VERSION}.yaml"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/secrets/do-spaces-credentials-sealed.yaml"
git commit -am "Adding Loki stack manifests for Flux CD"
git push origin

After waiting for a bit, let's check the configuration details of Prometheus using HelmRelease:

flux get helmrelease kube-prometheus-stack

The result you see resembles something like this:

OutputNAME                    READY   MESSAGE                                 REVISION        SUSPENDED 
kube-prometheus-stack         True    Release reconciliation succeeded        35.5.1          False

Now, let's find the 'READY' column in the output. If it says 'True', everything is good to go! Check the 'MESSAGE' column for updates on what's happening, and there's also a 'REVISION' number that shows the version of the setup. Sometimes, big setups like Prometheus take a bit more time, so don't worry if it's not super quick. Just hang in there and be patient.

Keep an eye on the 'MESSAGE' column; it will show 'Reconciliation in progress' while the HelmController is setting up the Helm chart. If anything goes awry, you'll see a different message explaining the issue, so be sure to check the status.
You can use a special command like this: 'flux get helmrelease <name> --wait' and it will keep you updated until it's done. Just know that your terminal might be tied up during this time, and it might take up to 5 minutes (though you can change this if needed).
If something doesn't work out, you can look in the Flux logs. Specifically, filter for messages related to HelmRelease, and that might give you clues on what went wrong

flux logs --kind=HelmRelease

If you still need more details and the Flux logs don't provide enough, you can use another command called 'describe'. This command helps you get a closer look at the HelmRelease setup using a tool called 'kubectl'. It's like getting a detailed report on what's happening with your setup. Here's how you do it:

kubectl describe helmrelease kube-prometheus-stack -n flux-system

Now, let's see if the special secret called 'prometheus-stack-credentials' was made too. You can check this by using another command, 'kubectl get secret prometheus-stack-credentials -n flux-system -o yaml'. This will let you peek inside the secret and see what's stored there. It's a bit like opening a locked box to see what's inside:

kubectl get secret prometheus-stack-credentials -n flux-system

At last, let's make sure everything's working with Prometheus! Here are some quick checks you can do:

To see the Prometheus dashboard, run: 'kubectl port-forward svc/kube-prom-stack-kube-prome-prometheus 9090:9090 -n monitoring'. It's like opening a window to peek inside.
For Grafana dashboards, try this: 'kubectl --namespace monitoring port-forward svc/kube-prom-stack-grafana 3000:80'. It's another way to look at things.
Lastly, to check the storage where Prometheus keeps its data, run: 'kubectl get pvc -n monitoring'. This is like checking the space where all the important stuff is stored.

Make sure to take a look at the '04-setup-observability' tutorial for additional info on checking how well your Prometheus setup is doing.

Now, onto the next step! We're going to create the instructions for setting up the Loki stack. And the cool part is, you'll let Flux CD do most of the work for you automatically. It's like giving your computer a set of instructions, and it takes care of the rest.

Easy Loki Setup with Flux CD

In this step, we'll get Loki up and running smoothly with Flux CD. Imagine giving your computer a clear guide to effortlessly install Loki on your cluster. Loki needs a secure spot to store its data, so we'll use a service like Cloudtopiaa Spaces for that. We'll also dive into keeping important info safe using Kubeseal CLI with Sealed Secrets Controller. Then, we'll explore how Flux CD cleverly uses a HelmRelease manifest to securely access the Cloudtopiaa Spaces credentials stored in Kubernetes Secrets. It's all about making sure Loki is set up securely and works like a charm!

Let's go step by step:

Start by going to the folder where you cloned your Flux CD Git repository. Check if you've set up the folders the way we explained.

Now, grab the Loki magic instructions we have stored. It's like getting a special recipe from a cookbook – we're fetching it from the Starter Kit Git place.

curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/repositories/grafana.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/grafana.yaml"

First, make sure you've shared a special key with your computer, as we explained in the part about exporting the Sealed Secrets Controller Public Key. If you haven't done that yet, make sure to replace any empty spots with the right information:

SEALED_SECRETS_PUB_KEY="<YOUR_SEALED_SECRETS_PUB_KEY_NAME_HERE>"
DO_SPACES_ACCESS_KEY="<YOUR_DO_SPACES_ACCESS_KEY_HERE>"
DO_SPACES_SECRET_KEY="<YOUR_DO_SPACES_SECRET_KEY_HERE>"
kubectl create secret generic "do-spaces-credentials" --namespace flux-system --from-literal=access_key_id="$ {DO_SPACES_ACCESS_KEY}" --from-literal=secret_access_key="$ {DO_SPACES_SECRET_KEY}" --dry-run=client -o yaml | kubeseal --cert="$ {SEALED_SECRETS_PUB_KEY}" --format=yaml > "$ {FLUXCD_HELM_MANIFESTS_PATH}/secrets/do-spaces-credentials-sealed.yaml"

The command we used consists of a few parts:

--namespace: This sets the place where the secret, containing Kubernetes access credentials, will be kept, like picking a specific folder.

--from-literal: This is about creating the secret by putting in the actual access key ID and secret access key, like noting down important information.

--dry-run=client: It's like practicing the command. It shows what the secret, with Kubernetes access credentials, would look like without actually saving it permanently. Think of it as double-checking your work before making it final.

4. Now, get the instructions for setting up Loki with Flux CD from the Starter Kit Git repository. It's like grabbing a step-by-step guide to make Loki work on your computer using Flux CD.

LOKI_CHART_VERSION="2.6.4"
curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/releases/loki-stack-v$ {LOKI_CHART_VERSION}.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/loki-stack-v$ {LOKI_CHART_VERSION}.yaml"

Now, take a look at the Loki setup instructions you just downloaded. You can open them in a text editor, like VS Code, which is like using a special notebook for computer instructions. Check for any blank spots marked with <> and fill them in with your specific information. It's a bit like personalizing a recipe to make it just the way you like!

LOKI_CHART_VERSION="2.6.4"
code "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/loki-stack-v$ {LOKI_CHART_VERSION}.yaml"

Note: Take a look at how the 'do-spaces-credentials' secret is used in the Loki setup instructions. It's like finding a specific item in a set of directions – this secret is essential for Loki to function correctly. Check the provided instructions to see how it's a key part of the whole process!

At last, save your changes in Git and share them with others. Think of it like saving your work and letting everyone else know about the updates you made.

LOKI_CHART_VERSION="2.6.4"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/grafana.yaml"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/loki-stack-v$ {LOKI_CHART_VERSION}.yaml"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/secrets/do-spaces-credentials-sealed.yaml"
git commit -am "Adding Loki stack manifests for Flux CD"
git push origin

After waiting for a bit, let's check the Loki setup instructions again. It's like looking back at the steps to make sure everything is going as planned.

flux get helmrelease loki-stack

The result you see is something like this:

OutputNAME         READY   MESSAGE                                 REVISION       SUSPENDED 
loki-stack         True    Release reconciliation succeeded        2.6.4          False

Check the 'READY' column – it should say 'True.' The 'MESSAGE' column will show how things are going, and there's also a number called 'REVISION' that represents the version of the setup. Sometimes, setups like Prometheus might take a bit longer, so just hang in there and be patient.

As the HelmController installs the setup, the 'MESSAGE' column will say 'Reconciliation in progress.' If something doesn't go as planned, you'll see a different message explaining what happened. Make sure to keep an eye on how things are going.

To wait until everything finishes, you can use a special command with '--watch' like this: 'flux get helmrelease <name> --wait.' It's like keeping an eye on the cooking timer.

Just remember, your computer might pause for up to 5 minutes in this mode, but you can change this time if needed. If things go wrong, you can check the Flux logs to find messages related to the setup using a filter.

flux logs --kind=HelmRelease

If you don't find enough details in the Flux logs, you can use another command. It's like asking your computer to explain things more. Just run 'kubectl describe helmrelease <name>' – it's a bit like asking for a detailed report on what's happening with your setup

kubectl describe helmrelease loki-stack -n flux-system

At the end, make sure the 'do-spaces-credentials' secret is created. You can check it by running 'kubectl get secret do-spaces-credentials -n flux-system -o yaml'. Think of it like double-checking to see if your computer has safely stored the secret information you provided.

kubectl get secret do-spaces-credentials -n flux-system

For more details on how to check if Loki Stack is working well, refer to the tutorial.

Now, let's move on to the next step! We're going to create the instructions for Velero, and Flux CD will take care of setting it up automatically. It's like giving your computer a list of steps, and it does the work for you!

Simple Velero Setup with Flux CD

Now, let's make Velero work smoothly with Flux CD. Think of it as telling your computer an easy recipe to automatically install Velero on your DOKS cluster.

Velero needs a safe place to keep backups, so we'll use a DO Spaces bucket. To make this happen, you'll need special credentials, but don't worry! We'll guide you on keeping them secure using kubeseal CLI with Sealed Secrets Controller. Then, we'll show Flux CD how to use these secure credentials to install Velero using the HelmRelease manifest. It's all about making Velero setup secure and easy!

Note: Before moving forward, just make sure that you've already set up the 'do-spaces-credentials' sealed secret on your DOKS cluster. You can find details on how to do this in the section where we created the Helm Release for Loki Stack

Getting Ready for Velero Setup

Start by heading to the folder where you saved your Flux CD Git repository. Make sure everything is set up correctly by checking the directories and making sure the FLUXCD_HELM_MANIFESTS_PATH environment variable is set. If you're unsure, refer back to the section where we explained how to clone the Flux CD Git Repository and set up the layout.

Next, grab the Velero HelmRepository instructions from the Starter Kit Git repository. It's like picking up a helpful guide to get Velero ready on your computer using Flux CD.

curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/repositories/vmware-tanzu.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/vmware-tanzu.yaml"

Create and encrypt a special space for Velero HelmRelease to use a secret code. Make sure you have the Sealed Secrets key exported, as explained in "Exporting the Sealed Secrets Controller Public Key" (replace <> placeholders accordingly).

SEALED_SECRETS_PUB_KEY="<YOUR_SEALED_SECRETS_PUB_KEY_NAME_HERE>"
DO_API_TOKEN="<YOUR_DO_API_TOKEN_HERE>"
kubectl create secret generic "do-api-credentials" --namespace flux-system --from-literal=do_api_token="$ {DO_API_TOKEN}" --dry-run=client -o yaml | kubeseal --cert="$ {SEALED_SECRETS_PUB_KEY}" --format=yaml > "$ {FLUXCD_HELM_MANIFESTS_PATH}/secrets/do-api-credentials-sealed.yaml"

Understanding the Command:

--namespace: Picture this as telling the computer in which specific area (like a room) of the Kubernetes system it should keep the secret.
--from-literal: Imagine you're instructing the computer, "Hey, create a secret and put this special code (your DO API token) inside it." This special code is used by Velero within the Kubernetes environment.
--dry-run=client: It's akin to practicing before you actually do it. This part helps you see what the secret looks like without actually storing it in the computer's memory within the Kubernetes system. It's like making a draft first.

Now, get the instructions for setting up the Loki stack from the Starter Kit Git repository.

VELERO_CHART_VERSION="2.29.7"
curl "https://raw.githubusercontent.com/digitalocean/Kubernetes-Starter-Kit-Developers/main/14-continuous-delivery-using-gitops/assets/manifests/fluxcd/helm/releases/velero-v$ {VELERO_CHART_VERSION}.yaml" > "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/velero-v$ {VELERO_CHART_VERSION}.yaml"

5. Next, take a look at the downloaded Velero HelmRelease instructions using a tool like VS Code. Make any changes you need. If you see <>, make sure to replace them with the right information.

VELERO_CHART_VERSION="2.29.7"
code "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/velero-v$ {VELERO_CHART_VERSION}.yaml"

You can understand how the secrets named do-api-credentials and do-spaces-credentials are used in the Velero HelmRelease instructions by checking the following part:

Remember, the secret information we talked about earlier is placed in a special area called environment variables (look for configuration.extraEnvVars in the instructions). Keep in mind that if you check the Velero deployment details, these values might be visible in plain text. On real-world systems, you can add extra security measures (like RBAC) to limit who can see this information, especially in important places like production environments.

kubectl describe deployment velero -n velero | grep AWS_ACCESS_KEY_ID

Lastly, save your changes in Git by committing them to the remote branch.

VELERO_CHART_VERSION="2.29.7"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/repositories/vmware-tanzu.yaml"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/releases/velero-v$ {VELERO_CHART_VERSION}.yaml"
git add "$ {FLUXCD_HELM_MANIFESTS_PATH}/secrets/do-api-credentials-sealed.yaml"
git commit -am "Adding Velero manifests for Flux CD"
git push origin

After a little while, please take a look at the Velero HelmRelease again.

flux get helmrelease velero-stack

The result you'll see should look something like this:

OutputNAME                 READY  MESSAGE                             REVISION    SUSPENDED 
velero-stack               True   Release reconciliation succeeded    2.29.7      False

Check the column labeled READY - it should show "True." You can also see the status of the process in the column called MESSAGE, along with a number called REVISION, which tells you the version of the Helm chart. Keep in mind that some setups may take more time to finish, like the Prometheus stack. So, just hang in there and be patient.

When the system is working to install the Helm chart, you'll see a message saying "Reconciliation in progress" in the MESSAGE column. If there's a problem, a different message will explain what went wrong. So, keep an eye on the status of your Helm release.

You can use a command like flux get helmrelease <name> --wait --watch to patiently wait until everything is done. Just note that your terminal will be occupied until it's finished or a default timeout of 5 minutes passes (you can change this with the --timeout option).

If things don't go well, you can check the Flux logs. Specifically, you can filter them to only show messages related to HelmRelease.

flux logs --kind=HelmRelease

If you don't find enough info in the Flux logs, you can use a command called "describe" with kubectl to learn more about the helmrelease. Here's how you can do it:

kubectl describe helmrelease velero-stack -n flux-system

Last but not least, confirm whether the special storage space called "do-api-credentials" was created in Kubernetes. If you want to see what's inside, you can use a command like kubectl get secret do-api-credentials -n flux-system -o yaml to check the secret contents.

kubectl get secret do-api-credentials -n flux-system

For more detailed information about making sure Velero is working correctly, take a look at the Velero tutorial. It will provide you with additional insights into checking Velero deployment status and how it functions.

Conclusion :

In this guide, you discovered the foundations of automating tasks using Flux CD in a GitOps setup. You set up Flux CD to release Helm charts automatically, deploying all the Starter Kit pieces effortlessly in a GitOps style. Plus, you added an extra layer of security by using Sealed Secrets to keep sensitive information safe for your applications.

Ready to explore more? Check out additional resources to deepen your understanding of GitOps and automation!